Repository navigation
feat(admin): add operations console for mirrors, workers, and NIP-05 - #895
Conversation
Avoid Redis client creation in worker factories when cache is unavailable, and flush async probe work under fake timers after ops heartbeats.
🦋 Changeset detectedLatest commit: d145049 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
[Medium impact] The PR appears safe to merge, with a non-blocking display issue in storage counts. Findings
|
Escape untrusted ops UI data, stop polling on logout, add NIP-05 paging, and make heartbeats/storage counts safer without blocking worker jobs.
Coverage Report for CI Build 39290635299Warning No base build found for commit Coverage: 73.787%Details
Uncovered Changes
Coverage RegressionsRequires a base build to compare against. How to fix this → Coverage Stats💛 - Coveralls |
chappie-daemon
left a comment
There was a problem hiding this comment.
Review: approve-level
Verified independently on the PR head: the 5 new admin-ops unit tests and the 8 relay-monitor worker tests pass, tsc reports no errors in any PR file, and biome lint (what CI runs) is clean on all new/changed source. The security-relevant claims check out: all five routes carry the identical adminRateLimitMiddleware + adminAuthMiddleware chain as their neighbors, the storage-counts query parser bounds kinds (max 20, 0–65535) and the time window (90 days) with a 5s statement timeout mapped to 504, the NIP-05 list clamps limit/offset and validates minFailures with everything parameterized through knex, the reverify endpoint schema-validates the pubkey and 404s/409s/502s correctly, and the dashboard escapes every interpolated value (escapeHtml/escapeAttr on the user-controlled NIP-05 identifiers included) — the #110 lesson visibly applied. Heartbeat publishing is properly failure-isolated so a Redis outage can't break the workers, intervals are cleared on exit, and the changeset is included this time.
One finding that should be fixed before merge:
The relay-monitor keepalive wipes the per-run heartbeat. publishWorker does a full setKey overwrite, and the 45s keepalive publishes a bare {type: 'relay-monitor', updatedAt} — no lastError, no run timestamps. So a failed probe run's lastError vanishes from the Operations dashboard within 45 seconds even though nothing recovered; the panel reads "Live" with no error until the next probe run, which may be far later. The static-mirroring worker doesn't have this problem (its keepalive republishes full state). The keepalive should re-publish the last known partial (kept in a member field) rather than a bare liveness ping — otherwise the worker-ops panel misleads exactly when it matters.
Further notes, non-blocking:
- Thin tests for the size. 1725 additions, 5 new unit tests. The pure functions (
sanitizeAdminOpsError,isAdminOpsHeartbeatStale,mergeMirrorRows/mergeWorkerRows) and the store's TTL/index logic are all easily unit-testable — and the keepalive finding above is exactly the class of bug a store-level test would have caught. - The heartbeat index set never shrinks. Dead
mirror:<index>/worker:<type>:<index>members are skipped on read (missing key → continue), but they accumulate inadmin:ops:heartbeat:indexforever — a slow Redis memory leak across config changes over a relay's lifetime. getAdminOpsStoreis a singleton that silently ignores thecacheargument on later calls. Fine with one cache per process, but an assertion would be cheaper than a future mystery.- Template checklist unfilled again — though the changeset made it in this time, which is the part that matters.
chappie-daemon
left a comment
There was a problem hiding this comment.
Review update: requesting changes
Revising my verdict on this one — approve was the wrong call, and the coverage question is what settled it.
The keepalive clobber isn't a polish nit: it silently wipes a failed probe run's error from the Operations dashboard within 45 seconds, which is the exact opposite of what this feature is for. And the new tests cover ~15% of the new backend code (zero on the store, the controllers, the sanitizer, the config merge) — so nothing in the PR's own suite guards the fix or would catch the next bug of the same shape. A real functional bug in the feature's core purpose, plus tests that can't see it, is request-changes.
What flips it back: have the relay-monitor keepalive re-publish the last known heartbeat state instead of a bare liveness ping, and add store-level tests covering publish/list/stale behavior plus the keepalive interaction. The pure functions (sanitizeAdminOpsError, mergeMirrorRows/mergeWorkerRows, isAdminOpsHeartbeatStale) are cheap to cover and would have caught this class of bug.
Keepalive republishes last probe worker state, reverify upserts only when the stored nip05 still matches, and add ops console unit tests.
Successful probe runs reset stored ops heartbeat errors while keepalives still refresh updatedAt from the last known run state.
Recreate the ops store when the cache instance changes so relay monitor unit tests do not share heartbeat mocks across cases.
a3fe2b3 to
6db1664
Compare
| showStorageError(payload.error ?? 'Storage count query failed') | ||
| return | ||
| } | ||
| renderStorage(payload.counts) |
There was a problem hiding this comment.
Storage counts hide their range
With Since and Until left blank, the server counts only the last 24 hours. This page discards the returned since and until values and shows an unlabeled storage count, so operators can mistake it for the total stored events. Show the effective date range beside the results, or make the 24-hour default visible before running the query.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Description
This PR adds an Operations area to the admin console so operators can inspect mirroring, background workers, NIP-05 verification state, and storage counts without SSH or direct DB access.
GET /admin/ops/mirrors,GET /admin/ops/workers,GET /admin/ops/nip05-verifications,POST /admin/ops/nip05-verifications/:pubkey/reverify,GET /admin/ops/storage/countsRelated Issue
Closes :- #816
Motivation and Context
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
Checklist: